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The explanation of the spectrum of quark and charged lepton masses and the Cabibbo- 
Kobayashi-Maskawa (CKM) quark mixing matrix is an outstanding challenge that requires 
physics beyond the standard model (SM). There are several general features that one would 
like to understand. Why is the heaviest known fermion a charge 2/3 quark, rather than 
a charge —1/3 quark or a lepton? Why does this heaviest known fermion, the top quark, 
have a mass that is comparable to the electroweak symmetry breaking (EWSB) scale M ew = 
v/V2 = 174 GeV, where v = 2- 1 ^G F 1/2 = 246 GeV? Why, in each generation, are the 
quarks heavier than the leptons and why, in the heavier two generations, is the mass of 
the charge 2/3 quark greater than the mass of the charge —1/3 quark? Why are neutrino 
masses, for which there is increasingly strong evidence, so much smaller than the masses of 
quarks and charged leptons? Recently, a new approach to fermion mass hierarchies has been 
studied, in which one assumes an underlying higher-dimensional spacetime and obtains the 
hierarchies from the localization of fermion wavefunctions at different points in the higher 
dimensions [0- @. Here we study effects of gauge interactions between fermions in this type 
of theory. We show that these effects are important and could help to explain the above- 
mentioned features of fermion masses. We use the notation Ui, di, and with % = 1, 2, 3 to 
refer, respectively, to u, c, t, d, s, b, and e, /i, r. 

Let us briefly describe the framework. For each generation, we denote the left-handed 
fermion fields as Qi and Lj for the quark and lepton SU(2) doublets and u°, df, and e\ L for 
the SU(2) singlets. Usual spacetime coordinates are denoted = 0, 1, 2, 3 and the n 

extra coordinates as y\\ for definiteness, the latter are taken to be compact. Generic fermions 
fields are denoted ^(x, y) = ip(x)x(y)- In the extra dimensions the gauge, Higgs, and fermion 
fields are assumed to have support in an interval < y\ < L. The d = 4 + n-dimensional 
fields thus have Kaluza-Klein (KK) mode decompositions. We shall work in a low-energy 
effective field theory (EFT) approach. The gauge fields extend over the interval < y\ < L, 
consistent with the observed universality of gauge-fermion couplings. A similar assumption 
is made for the Higgs field(s). The fermion wavefunctions are localized at different values of 
y\. This could occur in string theories §|, but here, for technical simplicity, we assume that 
it occurs in a field-theoretic manner. For example, for the case n = 1, before the inclusion 
of gauge interactions, consider the action for the quarks: 

Soc Jd 4 xdy ^Q^l-d + L^-M^Q^Y, E %(^7 • ^ + L 1 / 2 ^ - A^ojJ^ 

L i i f=u,d 

^ T Q .C^ u oH u + h.c) 

where 7 • d and C5 are the five- dimensional Dirac operator and charge conjugation matrix. 
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In (|), in the SM, H d = W and H u = H\ with H being the SM Higgs and H = ia 2 H*; more 
generally, H u and H d may be independent Higgs fields, as in the minimal supersymmetric 
SM (MSSM). A similar formula holds for leptons. The proportionality factor in (|l|) is chosen 
to yield a canonically normalized 4D action. If y) has the usual kink solution $ = 
$o tanh(/zy), this traps each fermion to a domain wall at yj i = £f i = —M.qjJ^ 2 @ with 
localization length //~ x . Starting with Dirac fermions in the five- dimensional space, this 
trapping mechanism yields a chiral theory in which only left-handed fermions are trapped 
on the physical 4D domain wall. In the standard model, the fermions trapped to the physical 
domain wall are then Q iy u^, d^, L iy and i = 1,2,3. More generally, one considers the 
possibility of fermion localization with n > 1. In the quark sector, for N g generations, one 
can choose the values of the n(3N g — 1) coordinate differences for the wavefunction centers 
£f t to fit the 2N g quark masses and (N g — l) 2 parameters determining the CKM matrix, i.e. 
the N p = Ng + 1 physical parameters. 

If << L; then, to a good approximation, a generic fermion wavefunction is a gaussian 
of the form ^/(x,y) = ip{x)x{y) with x{v) °t e~^ 2( - y ~^ 2 ^ 2 , where {y — £) 2 = ^a=i(^ ~ ^) 2 - 
Performing the integration over y, one obtains the 4D quark Yukawa couplings 

S Y = K dfj ] / d 4 xip djR (x)ipQ iL (x)H d (x) + K u,i? / d A xip U]R (x)i) QiL (x)H u (x) + h.c. 

(2) 

where 

(AD) _ -f(i Qi -i s flA , s 

f,ij ~ ' H f-'J ■ ( - )! 

Thus, separations \£q. — £fc\ that are moderate in units of produce a strong gaussian 
suppression of fermion overlaps and hence of the associated 4D Yukawa couplings. Similar 
comments apply for the charged leptons. Since the purpose of this type of model is to derive a 
hierarchy without putting it in initially, one takes the Hf^j ~ 0(1). By choosing the different 
separations, one can account for the observed fermion mass and mixing angle hierarchies in 
terms of the relative locations of fermions in the extra dimensions and further explain proton 
longevity and the weakness of flavor- changing neutral current processes. This type of model 
involves three general length scales: L, fi , and, since it is a low-energy effective field theory 
(EFT), a high-energy cutoff, A. Besides the \T X « L condition, one requires /i << A for 
the self-consistency of the theory. Possible values for these parameters, consistent with 
experimental constraints are sim L~ x ~ 100 TeV, [ijL~ x ~ 30, and A//i ~ 20 |l|]- [||], |10 



We proceed to incorporate the SM SU(3) x SU(2) x U(l)y gauge interactions. Our 
notation is indicated by the covariant derivative on quarks, = <9 M — ig^O^ — ig^A^P^ — 
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i(g'/2)(Y L P L + Y R P R )B„ where C, = El=x^(K/2), A, = £Li A»(r a /2), C* AJ, and 
£> M are the SU(3) color, SU(2) L , and weak hypercharge gauge bosons, and Pl,r are chiral 
projection operators. Thus, g' j g% = tan^jy and g' = (?>/h) l l 2 gi, where g^ i — 1,2,3 are the 
couplings that would unify (= g) at a high mass scale in a grand unified theory |Tl . 



Because the fermions are localized in the higher dimensions, with the wavefunction fac- 
torization given above, they are essentially static as functions of y, so we need only consider 
the Coulombic interaction between them. Since /z^ 1 << L, the Coulomb potential is that for 
the full d = 4 + n dimensional space. Also ji » L^ 1 » M ew , so (i) the relevant gauge fields 
are massless on this scale, and (ii) the color contribution is perturbatively calculable to good 
accuracy. The Yukawa operators dj^QuHd and UjRQnH u yield, via the Higgs vevs, the 
bilinears dj^dn and UjRUn. Each of these bilinears involves only the one SU(2) nonsinglet 
fermion, Qn, so only the (vectorial) SU(3) C color and (chiral) U(l)y hypercharge interactions 
between these fermions contribute. The distance between the centers of the wavefunctions 
of the fermions fn and f? L at (x,£fj and (x,£f?) is df^j = \£f t — £f?\. Using Yq l = 1/3, 
Yl l — — 1, Yf R = 2Qf, the Coulomb interaction energy of fn and fjR is 

VcUv) = (4) 

with \y\ = df^j, where aj = Cf/A 3+n , Ai = 27r £ / 2 /r(£/2) is the area of the unit sphere S e , 
and 

Cu = £03 + J^df , (5) 

Cd = ^g 2 3 - ^g\ , (6) 
c e = \g\ • (7) 

For example, in the operator UjRUn one has two fundamental representations of color SU(3) 
contracted to a singlet, so the color interaction is attractive, and the coupling constant 
dependence is — (4/3)gf |l|]. Since the hypercharges Y of the un, and UjR are 1/3 and 



—4/3, the hypercharge interaction is (1/3) ( — 4/3) (c?') 2 = ~~ (4/15)(7f , as given above. The 
normalization in @ satisfies the requirement that as \y \ increases through the value L beyond 
which the effective dimension of spacetime is 4, the potential matches the usual 4D Coulomb 
potential. Note the general inequality 

c u > c d >c e . (8) 
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If one makes the additional assumption of gauge coupling unification gi = g at some scale 
Mjj with ji > Mjj > L" 1 , then for distances d such that A -1 < d < My 1 , c u = (8/5)g 2 and 
c d = c e = (Q/5)g 2 . 

Before inclusion of gauge interactions, the suppression in @ can be treated as the result 
of the tunnelling of each fermion to the midpoint of the line joining £f f and Efc, located 
at a distance d/,y/2 from each Jl|. The WKB tunnelling amplitude |H| for each fermion 
is exp(— J'q 1 ' 1 ^ 2 V(r)dr) = exp(— (fidf^j) 2 /8), yielding, for the total suppression, the factor 
exp(— (fidf^j) 2 in agreement with (|3|). 

We now consider the effect of the Coulomb interactions. Since Ljdf^ values are reason- 
ably large, the effects of the boundary conditions on the gauge fields, e.g., image charges, 
are small, and we neglect them. Given that the fermions have the same localization 
lengths, the tunnelling can be treated in a symmetric way for each pair. The full po- 
tential energy for at a distance r outward from its wavefunction center toward /? is 
Vf(r) = Vt r ap(r) + Vcoui(df t ij — r), where Vt ra p(r) = fi 2 r. The resultant restoring force in 
the f direction is F = —fi 2 + cif(n + l)L n /(d/y — r) n+1 . The most dramatic effect occurs if 
F(r = 0+) = -fi 2 + a f {n + l)L n /(d Uj ) n+2 is positive, i.e., 

(fid M ) 2 < a/ (n + l)(— -J ; (9) 

here, the Coulomb attraction overwhelms the trapping potential and causes fi and fj to have 
wavefunctions that are centered at the same location, at least to within the distance A -1 
down to which the EFT applies. Hence in this case there is no suppression of the Yukawa 
coupling ^fij\ so that, if the higher-dimensional coupling Kf^j ~ 0(1), then the resultant 
fermion mass is of order M ew . Let us first concentrate on the third generation and neglect 
small mixing effects. We further focus on the n = 2 case since a successful fit was found for 
this case in [[| (before the inclusion of gauge interactions). This fit yielded fJ,d U:33 = 0.9005, 
fidd,33 = 3.00, and /4<4,33 = 3.15, with fiL ~ 18. We use the illustrative gauge-coupling 
unification value g% = g with g 2 /(4ir) ~ 0.04, so that a u = 0.030 and = a e = 0.023. 
Then (fidu^) 2 = 0.81, which is smaller than the RHS of (§J), viz., 46, while (/ic/^33) 2 > 
RHS@ for / = d, e. Thus, with these input values, the Coulomb interactions cause tt 
and t c L wavefunctions to be centered essentially on top of each other, but do not overwhelm 
the trapping of other fermions to their domain walls. Generalizing, we can say that if a 
fermion has /ic/j i33 < 0(1), i.e., is moderately heavy, then, for a value of /iL ~ 20 that 
is phenomeno logically acceptable, LHS(|) < RHS(||), so that the Coulomb interaction can 
dominate over the trapping interaction with $ and cause the chiral components of this 
fermion to lie essentially on top of each other, leading to m/ ~ M ew if ~ 0(1). For 
plausible input values, this can happen for a charge 2/3 quarks while the charge —1/3 quarks 
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and leptons remain trapped on their domain walls. This could thus help to explain the fact 
that the heaviest known fermion, and the only fermion with a mass comparable to the EWSB 
scale M ew , is a charge 2/3 quark, rather than a charge —1/3 quark or a charged lepton and 
thus could provide deeper insight into properties (i) and (ii) in the abstract. 

Although we have discussed this Coulomb-induced collapse in the model of 0, we also 
note that, more generally, gauge interactions could also be relevant to models with dynamical 
EWSB involving multifermion operators in which a (//) condensate forms. We suggest that 
these gauge interactions and the inequality (H) could explain why in such dynamical EWSB 
models it is the (it) condensate that forms rather than a (bb) or (fr) condensate. 

Just as a WKB approximation can be used to infer the result (§) before inclusion of 
the Coulomb effects, so also it can be used to calculate the latter effects. We next do this 
for fermions for which this attraction does not overwhelm the trapping to domain walls. 
Consider the symmetric path where and /? each tunnel a distance r toward each other 
from their respective centers (x, £f t ) and (x, £fo) so that they are a distance 26/^ = dfjj — 2r 
apart. The classical turning points {rt)f,ij occur where the total potential energy 



^'^' (J^ (10) 

vanishes. We have e^y = (1/2)^/^ — (r t ) /,y and define r)f } ij = 2e/ ! y/d/ i y. Then the equation 
for the turning point becomes 

'/;.;/(' -'/./,,) t>r (ii) 

where bf = af(jidf^j)~ 2 {L/df^j) n . Since the mutual tunnelling by the fermions does not 
have to proceed further than to the midpoint between them, we are only interested in the 
range < rjf^j < 1. For this range, the LHS of ( |TTD increases from to a maximum value 
bf, m — (n+ l) n+1 /(n + 2) n+2 at (?7/,r,) m = (n + l)/(n + 2) and then decreases to again 
at T)f t ij = 1. If bf < bf ;Tn , ( fn| ) has physical solutions. For bf = bf :Tn , there is a unique 
physical solution, r)f t ij = (r]f ti j) m . For < bf < bf tmax , the two turning points are given by 



r 



1 - 7ffg)df t ij/2 with < 77) I < (n + l)/(n + 2) < rff^ < 1. The relation between 



the higher- dimensional and 4D Yukawa couplings is then given by 

where the overlap factor is, in the WKB approximation, Wf^j = exp(— Jf,ij), with 

rf.h 

Jf,ij = / 3 V tot (r)dr 
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mj - 'if. 



(^ui\ 2 l2 _ (r) m , «x, a _i fly 

{ 2 ) I 2 fo/* + WJ ^w^U^J 



(13) 



The integral extends over the classically forbidden region between the two turning points 
|T3| . For the n = 2 parameters above, we find that the gauge interaction increases the 



overlap factors 11^,33 and u> ei 33 relevant for m;, and m T by 22 % from 0.77 to 0.98 and from 
0.76 to 0.93, respectively. Thus, gauge interactions have a significant enhancement effect 
on the wavefunction overlaps and hence Yukawa couplings. Although we have concentrated 
on the case g% = g\, the more general case g% > g\ would lead to further enhancement of 
the masses of the charge 2/3 and —1/3 quarks relative to those of the charged leptons. Of 
course, this is not an ah initio calculation of the fermion mass spectrum, since it depends 
on initial inputs for the relative distances df^j. What our calculations show is that gauge 
interactions, together with the inequality (Q), could help to explain why the quarks of a 
given generation are heavier than the charged lepton and why, at least for the higher two 
generations, m Q=2 / 3 > m Q= _i/ 3 , i.e. property (iii). 

For the first generation, given the smallness of m u and |14| , a model for these masses 
should take account of the off-diagonal quantities ij = 12, 21. Indeed, if the N g — 1, 2 

subsectors of the mass matrices for the charge 2/3 and —1/3 quarks have the form (after 
allowed rephasings so that is real and positive) 

M«=(>°, S) <"> 

V^l ^22 / 

( f) ( f) 

where \ A\ij \/A^ « 1 for ij = 12, 21 and ~ means a negligibly small entry, the eigenvalues 
have the form A^ /} ~ 4? and l^l - l^iMflM? (with = m c , \f ] = m s , \X ( ^\ = 
= m d ). Since A^ ~ m c » A^ — rn s and since these enter in the denominators 
of the expressions for the lighter eigenvalues, if A^' and A\j\ ij = 12,21 are not too 
different, this seesaw effect could accomodate the fact that m u < m^. That is, these lightest 
quark masses could arise primarily by mixing, and hence could avoid the generic pattern 
m Ui > > m ei for the heavier two generations i = 2,3. 

An important comment concerns the calculability of these gauge interaction effects. The 
fact that fermions with stronger gauge interactions (color and U(l)y or U(l) em ) are more 
massive is very suggestive. Yet conventional attempts to explain this via radiative correc- 
tions in usual quantum field theory encounter the obstacle that perturbative gauge boson 
couplings preserve chirality and cannot generate masses from originally massless fermions. 
Once fermion masses are generated, by the Higgs mechanism or in some other way, radiative 
gauge boson corrections modify the tree-level masses; however, these corrections are diver- 
gent, so that the physical masses are arbitrary numbers which are inserted to fit experiment. 
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In contrast, our effects are calculable and finite. Even in the case where they overwhelm the 
domain- wall trapping, and hence there can be sensitivity to A, this just results in Wf^j ~ 1. 

We next consider neutrinos. In the minimal approach, where one avoids electroweak- 
singlet neutrinos, the Majorana mass term vJ L C(ML)ijVjL would arise from the operator 

= TrYl hij{za P e bq + e aq e bp )Li a C d L b j H*H* + h.c. (15) 

My — 

hj 

(where a,b,p,q are SU(2) indices) via the vevs of the Higgs. Here, M x could be ~ 0(A). 
Since Lf and L b are coupled to I = 1 and since they have the same hypercharge (Y = — 1), 
both the SU(2) and U(l)y Coulomb interactions are repulsive for the Lf , L b pair, separated 
by the distance diL,ij- This would tend to move the corresponding wavefunctions farther 
apart and thereby suppress the wavefunction overlap, naturally leading to small 4D operator 
coefficients. For the case i = j, the Coulomb self-energy is especially large, ~ fi^A, k = 
1,2. This would strongly suppress the diagonal contributions to the left-handed neutrino 
Majorana mass matrix. In turn, this could naturally lead to large mixing. Given that the 
leptonic mixing matrix U = U U U^, if U e has only small mixing, a (2,3) subsector of the vn 
Majorana mass matrix Ml (which always satisfies (Ml)^ = (M£)ji) of the form 

< M ')=(c° ~o) < 16) 

will yield maximal z/ M , u T mixing, in agreement with the atmospheric neutrino data []16| , |17| . 

Some of the features that we have found may well transcend the specific EFT approach 
used here. Indeed, the latter leaves a number of open questions. Why are there three 
generations of standard model fermions standard model with light neutrinos? What are the 
implications of the fact that the theory is nonrenormalizable in d > 4 dimensions? What 
further insights can these models give to EWSB (e.g. |Tj|) ? What physics gives rise to the 
restriction of the gauge and matter fields to M x [0, L] n , where M is Minkowski space, and 
to the localization of the fermion wavefunctions? The latter problem may well be related 
to the open string/brane construction of gauge theories. We speculate that the localization 
of fermions and resultant gaussian profile might be achieved without introducing the $ field 
and higher-dimensional Yukawa couplings and masses and will return to this issue later, 
together with further study of quark and lepton masses and mixing. 

In summary, we have shown that the inclusion of gauge interactions in models with 
fermion wavefunctions separated in extra dimensions has important consequences and could 
help to provide an explanation of several of the most basic features of the known fermion 
masses. This explanation is pleasingly minimal in that it makes use only of the established 
SM gauge transformation properties of the known fermions, albeit in a new context. 
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